1. Field of the Invention
The present invention relates to an image-processing device and an image-processing method, an image-pickup device, and a computer program that are provided to process image data including data on the image of a subject such as a person, and particularly relates to an image-processing device and an image-processing method, an image-pickup device, and a computer program that are provided to detect the subject-image data included in the image data and perform image processing appropriate for the subject-image data.
More specifically, the present invention relates to an image-processing device and an image-processing method, an image-pickup device, and a computer program that are provided to process image data including data on at least two subject images, and particularly relates to an image-processing device and an image-processing method that are provided to determine the importance of the subject images on which data is included in the image data, an image-pickup device configured to perform imaging processing based on the importance-determination result, and a computer program.
2. Description of the Related Art
Cameras have had a long history, as devices for recording visual data. Recently, digital cameras configured to digitize an image captured by a solid-state-image sensor including a charge-coupled device (CCD), a complementary metal-oxide semiconductor (CMOS), etc., are widely available in place of silver-halide cameras configured to make an exposure by using a film and/or a photosensitive plate. The digital camera can store data on a digitally-encoded image in a memory, and perform image processing and/or an image management by using a computer. Furthermore, the digital camera is free of a trouble occurring due to the film life.
The basic camera works of the silver-halide camera are the same as those of the digital camera. That is to say, first, a photographer brings a subject into focus, adjusts the aperture of the camera, that is, the exposure amount, performs framing, so as to form an appropriate composition including the subject image, and presses down the shutter of the camera. For reducing hand-shake, the photographer grasps the main body of the camera and/or installs the camera on a tripod.
As automation technologies relating to the above-described camera works have progressed, the possibility of failing to make an exposure has become significantly low even though the user of a camera is inexperienced in camera operations. For example, the auto-focus (AF) function, the auto-exposure (AE) function, the hand-shake-correction function, the auto-shutter function, etc. have been achieved. In the case where the digital camera is used, AF processing and/or AE processing can be performed not only by using an optical system, but also by performing digital processing. Further, image-quality adjustment is performed for data on a photographed image through digital processing, such as applying a white-balance gain to data on a picked-up image through auto-white-balance (AWB) processing, so that an appropriate color status can be reproduced.
The auto-focus function, the image-quality-correction function, etc. should be used for any of at least one subject image shown in an image frame. In former times, the user of the camera manually determines an important subject. Recently, imaging technologies achieved by using face-detection technologies have been introduced. According to the face-detection technologies, a face image is detected from a picked-up image, and the face image is automatically brought into focus and/or image-quality correction is automatically performed for the detected face image, for example.
Here, a method of using data on an average face image, as template data, comparing the template data to an input-image signal, so as to determine whether or not the template data matches with the input image signal, is widely known in the art, as a method of detecting face-image data from the signal of a picked-up image (e.g., see Japanese Unexamined Patent Application Publication No. 2003-271933 (Paragraphs 0051 to 0053 and FIG. 6)). Further, when a picked-up face image is inclined, it is difficult to correctly detect the face image. In that case, therefore, the angle to which the face image is shown is detected, the face image is rotated according to the detected value, and matching processing or the like is performed. According to a method of detecting the angle to which the face image is shown, a center line of the face image is detected, and a degree indicating the probability that the detected center line corresponds to the true center line of the face is calculated based on the right-to-left symmetry of the face. If the calculated degree of the detected center line is high, the angle of the detected center line is determined to be the angle to which the face image is shown, and data on the angle is output (e.g., see Japanese Unexamined Patent Application Publication No. 9-171560 (Paragraphs 0032 to 0044 and FIG. 4)).
Further, a digital-camera-image-pickup device configured to press down a release button configured to photograph a subject in response to detection of a face image from within an image signal obtained via an optical system has been proposed, as an image-pickup technology achieved by using the face detection (e.g., see Japanese Unexamined Patent Application Publication No. 2003-92700).
Further, an electronic camera configured to detect the image of the face part of a subject from within a picked-up image and perform image processing for a detected face image, so as to reduce the red-eye phenomenon, has been proposed (e.g., see Japanese Unexamined Patent Application Publication No. 2005-86271).
As the accuracy of the face detection increases, the camera-work-automation technologies and/or the image-processing technologies achieved by using the face detection become more convenient and effective for the user. Namely, if the function of determining whether or not a detected face image is important for a photographer is provided, it becomes possible to perform focusing and/or image-quality correction for a necessary face image with stability.
For example, an image-pickup device resistant to the movement of a person and/or hand-shake has been proposed (e.g., see Japanese Unexamined Patent Application Publication No. 2005-318554). In the image-pickup device, image data is stored in memory space A, as data compressed under the RAW-compression system, video data generated by subjecting the image data to γ correction, interpolation, matrix conversion, etc. is stored in memory space B, and a face area is detected within a short time period based on image data stored in the memory space B with a capacity smaller than that of the memory space A.
Further, an electronic still camera configured to put the eyes of a subject into focus so that the entire face of the subject is brought into focus, and determine the subject depth so that two ranges of 10 cm, where the two ranges are separated by the focal distance, are roughly brought into focus has been proposed (e.g., see Japanese Patent No. 3164692).
In actuality, however, the detected face image is not necessarily the subject of the focusing and/or the image-quality correction. Therefore, the focusing and/or the image-quality correction may be performed for a detected face image which is not the subject of the above-described processing. Particularly, when the images of at least two subjects are produced in a picked-up image, it is difficult to determine who should be put into focus (that is to say, which subject should be determined to be the main part).
By intuition, it can be estimated that the importance of a subject image shown near the center of a picked-up image and/or a large subject image shown in the picked-up image is high, and the subject image is the main part. Therefore, after specifying the subject image shown in the picked-up image through the face-detection processing or the like, a priority is given to the subject image based on the distance from the center of the picked-up image to the subject image and/or the size of the subject range. Subsequently, the main-part determination can be made.
For example, a face-importance-determination device configured to determine the importance of a subject for a photographer with precision by acquiring the face-size information and the face-position information detected from an image signal, calculating the first importance factor based on the size information, calculating the second importance factor based on the position information, and calculating the definitive importance based on the first and second importance factors has been proposed (e.g., see Japanese Patent Application No. 2006-20137, where the right thereof had already been transferred to the applicant).
Further, an image-pickup device configured to store information about the feature of an object in advance, detect face images from image data, detect a detected-face image matching with predetermined feature data, and put the detected face image into focus has been proposed (e.g., see Japanese Unexamined Patent Application Publication No. 2002-333652). The above-described image-pickup device stores information about the object features and information about priorities assigned to the object features. If data on a plurality of persons is included in image data, the above-described image-pickup device makes matching determination in decreasing order of the object-feature priority.
Further, a focus-adjustment device configured to detect data on the eyes of a person whose data is included in image data, set a focus frame based on the position and/or size of each of the detected eyes, detect the focal-point position for each of the focus frames, and determine the nearest focal-point position of the focal-point positions to be the definitive focal-point position has been proposed (e.g., see Japanese Unexamined Patent Application Publication No. 2005-128156). In that case, the main-subject determination is made according to whether or not the detected focal-point position is the nearest focal-point position.
Further, a digital camera configured to easily perform face-image detection has been proposed (e.g., see Japanese Unexamined Patent Application Publication No. 2006-33437). When performing the face-image detection by using a through image, the digital camera increases the brightness of the through image so that the brightness of the through image becomes higher than that of the through image shown when the face-image detection is not performed so that the face-image detection can be easily performed. In the above-described digital camera, the frame of the largest face image and/or the nearest face image is determined to be an AF area when the images of at least two faces are detected.
Further, an image-pickup device configured to adjust the focal point of a photographic lens so that a face image shown at the shortest distance from the image-pickup device of all of detected face images is brought into focus, produce the image of a frame surrounding the face image brought into focus on a liquid-crystal display, and move the frame image to another face image through the user operation performed by using a cross key or the like when the face image surrounded by the frame image does not show the face of a desired person has been proposed (e.g., see Japanese Unexamined Patent Application Publication No. 2006-145629). Basically, in the image-pickup device, the main-subject determination is made according to whether or not a detected face image is closer to the image-pickup device than the other detected face images are.
However, when photographing a person and/or an animal such as a pet, the position and/or size of the above-described subject images shown in a picked-up image is not fixed, but changed every moment. That is to say, in a through image and/or video, the result of determination of a subject image which is the main part is changed with frequency so that the auto-focus operation and/or the image-quality-correction operation becomes unstable in the time direction.